Lower and upper bounds on the minimum mean-square error in composite source signal estimation

Performance analysis of a minimum mean-square error (mmse) estimator for the output signal from a composite source model (CSM), which has been degraded by statistically independent additive noise, is performed for a wide class of discrete as well as continuous time models. The noise in the discrete time case is assumed to be generated by another CSM. For the continuous time case only Gaussian white noise, or a single state CSM noise, is considered. In both cases, the mmse is decomposed into the mmse of the estimator which is informed of the exact states of the signal and noise, and an additional error term. This term is tightly upper and lower bounded. The bounds for the discrete time signals are developed using distribution tilting and Shannon’s lower bound on the probability of a random variable to exceed a given threshold. The analysis for the continuous time signals is performed using Duncan’s theorem. The bounds in this case are developed by applying the data processing theorem to sampled versions of the state process and its estimate, and by using Fano’s inequality. The bounds in both cases are explicitly calculated for CSM’s with Gaussian subsources. For causal estimation, these bounds approach zero harmonically as the duration of the observed signals approaches infinity.